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WWC Study Reports are intended to support decision making; neither the What Works Clearinghouse 
(WWC) nor the U.S. Department of Education endorses any interventions. No single Study Report 
should be used as a basis for making policy decisions because (1) few studies are designed and 
implemented flawlessly and (2) all studies are tested on a limited number of participants, using a limited 
number of outcomes, at a limited number of times, so generalizing from one study to any context is very 
difficult. To highlight these issues, the WWC Study Reports describe in detail the specifics of each 
study, focusing primarily on studies that provide the best evidence of effects (randomized controlled 
trials). Systematic reviews of the evidence will be conducted to extend the results of the individual 
studies. 
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What Is this Report About? 

The Expert Mathematician (version 3.0), a 
middle school math curriculum, was developed 
from LogoWriter, a computer program that the 
developer claims is designed to help children 
learn independently through exploration. 
According to J. J. Baker, the Expert 
Mathematician is designed to provide a creative 
environment that is intended to foster a child’s 



higher learning skills. It contains 196 lessons. 
This WWC Study Report reviews a study that 
compares the effects of The Expert 
Mathematician with those of a more traditional, 
teacher-directed curriculum (Baker, 1997). This 
WWC report summarizes the study and reviews 
its strengths and weaknesses. A more detailed, 
technical version of this study report is available 
here (PDF). 



The What Works Clearinghouse (www.whatworks.ed.gov) was established in 2002 by the U.S. 
Department of Education’s Institute of Education Sciences to provide educators, policymakers^ 
researchers, and the public with a central and trusted source of scientific evidence of what works in 
education. Please email all questions and comments to info@whatworks.ed.gov. The What Works 
Clearinghouse is administered by the U.S. Department of Education through a contract to a joint venture 
of the American Institutes for Research and the Campbell Collaboration. 
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How Was the Study Conducted? 

The study participants were 8th-grade students 
from a suburban middle school in St. Louis, 
Missouri. Most of the students were from low- 
income families and qualified for free or 
reduced-priced lunch. All but three of the 
students were white. The students varied in their 
ability; none were in special education. 

At the beginning of the study, 90 students were 
randomly divided by a computer algorithm into 
either the intervention or control group. The 
students were then divided randomly again, 
through a flip of a coin, into classes. There were 
two intervention classes and two control classes. 
Each class had about 23 students. 

In the Expert Mathematician (intervention) 
class, students worked individually or in pairs 
using the printed materials and the computer to 
learn a new procedure in order to solve a math 
problem. Classes were conducted every other 
day for one school year. Sessions were 85 
minutes long and included one to two math 
lessons. 

The outcome measure relevant to this WWC 
report was the Objectives by Strands test 
(sometimes referred to as “Mathematical 
Concepts and Applications Survey” by the 
author), which was used to measure the 
students’ mathematics achievement before and 
after the intervention. As reported by Baker, the 
test was developed by a large urban school 
district, and it contained 78 multiple-choice 
items. 



What Did the Study Find? 

In one of several analyses, Baker (1997) found 
that there was no significant difference in gain 
scores between students in the Expert 
Mathematician and the control groups when 
controlling on pretest. (See Figure 1.) However, 
Baker (1997) did report that the average gain 
scores of the students in the Expert 
Mathematician classes were similar to the gain 
scores of students in the control group on the 
Objectives by Strands test. In this study, natural 
student groupings (classrooms, schools, etc.) 
may have affected findings. Although the 
author’s analysis does not address this grouping 
problem, the author does not report significant 
positive findings, so the impact of groupings on 
findings is likely minimal. 



Figure 1. Impact Reported by Baker 3 




Expert Mathematician (n =34) 

(n =36) 



3 Confidence intervals were computed by the WWC. 

b The intervention group scores were not significantly lower than the control 
group scores (p< .05). 
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WWC Study Ratings 3 
Baker (1997) 


Causal Validity: Meets WWC Evidence Standards, a Randomized Controlled Trial with No 

Randomization, Attrition, or Disruption Problems 

Participants were randomly assigned to the intervention and control groups. There was a significant difference 
between groups on the Objective by Strands pretest scores; but this was addressed by the study author. There was 
severe overall attrition, but it did not affect the initial equivalence of the groups analyzed. Other than the pretest, 
no extraneous events were identified that appeared to be confounded with the intervention’s effect. 


Other Study 

Characteristics Study Rating 


Study-Specific Information 


Intervention Fidelity 


•• 


The intervention is well defined and implemented and meets the 
definition of Middle-School Math. 


Outcome Measures 


•• 


The test appeared to be appropriately aligned with the intervention. The 
reliability calculations are based on 62 survey items, although it was 
reported that the survey contained 78 items; however, this was not a 
significant issue. 


People, Settings, and 
Timing 


• 


The number of participants and settings was too small and homogeneous 
to allow generalization to the full range of people and settings that are the 
target of Middle-School Math interventions. The Objectives by Strands 
test was administered at the beginning and at the end of the study. 


Testing within 
Subgroups 


• 


The intervention effect was tested across the entire sample but not within 
important variations in settings. 


Analysis 


• 


The results were analyzed at the level of individual students, which 
matches the unit of randomization. In this study, natural student groupings 
(classrooms, schools, etc.) may have affected findings. Although the 
authors’ analysis did not address this grouping problem, the author does 
not report significant positive findings, so the impact of groupings on 
findings is likely minimal. The sample size was too small to allow for a 
precise estimate of effect. 


Statistical Reporting 


•• 


The author reports unadjusted means and standard deviations of the 
outcome measure, Objectives by Strands test, as well as the sample sizes 
for both the intervention and control groups, so an estimate of its effect 
could be calculated. 


Summary of Results 

There was no statistically significant difference in student performance between students receiving the Expert 
Mathematician curriculum and those in the control groups. Baker reported negligible differences between the 
Expert Mathematician intervention and control groups. The outcome measure is sufficiently reliable, based on the 
author’s report of its reliability, but the sample size was relatively small. In addition, the sampled participants and 
settings were restricted. 


Note. • • Fully meets criteria; • Meets minimum criteria; X Does not meet criteria. 
a For more information on the criteria used to rate this studv, see the WWC Studv Review Standards. 
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How Can You Find Out More? 

■To learn more about this study, read the 
detailed report (PDF) or the original study (PDF). 

■ To see reports on other studies of Middle 
school Math curriculum . 

■ Cost information: not available. 

■ Intervention developer contact 
information: 

The Expert Mathematician at: 
www.expertmath.org ; 
email: frstprin @ mninter.net ; 
telephone: (612) 872-6741. 



Report Production 

Date created: June 30, 2004 

■ The study was reviewed under the first of 
three waves (middle, elementary, and high 
school) under the topic — Curriculum-Based 
Interventions for Increasing K-12 Math 
Achievement. 
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